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Summary 

Aims: Considerable variability in (paediatric) trauma care has been reported. We wanted to 
audit current practice in Flanders (Belgium). 

Methods: The PENTA network prospectively collected data on paediatric trauma patients in a 
representative sample of Flemish hospitals during 2005. All cases with an ISS >13 and sufficient 
data availability were withheld for panel evaluation (n = 92). Two trained experts reviewed 
the medical care provided in the first hours after trauma, based on available evidence and 
existing universal guidelines. 'Defaults’ were only withheld as such if there was 100% consensus. 
At random, about 25% of cases were also reviewed by two other experts in order to assess 
interobserver variability. 

Results: In the 92 cases, 264 defaults were recognised. 25.4% of all defaults were thought to 
have a direct impact on the individual patient’s outcome. Specific difficulties were observed 
with, e.g. cervical spine management (18/82 relevant cases), pC0 2 and global respiratory 
management (38/92), fluid management (29/92) and analgesia (27/89). The agreement 
between the two panels was good for defaults identified (crude agreement 74.8%), yet only 
fair for the presumed impact on outcome (crude agreement 58.3%). 

Conclusions: We audited paediatric trauma care in Flanders and identified several problem 
areas (often in basic areas of paediatric life support). The inherent degree of interobserver 


7 A Spanish translated version of the summary of this article appears as Appendix in the final online version at 
doi: 10.1016/j. resuscitation.2008.04.029. 
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variability does not diminish the importance of these findings. More performance-based teaching 
and timely recertification may have a positive impact on the quality of the care delivered. 

© 2008 Elsevier Ireland Ltd. All rights reserved. 


Introduction 

"The profession can no longer hold privileges against 
both public and patient criticism without demonstrating 
clearly that we have the capacity to be critical of our 
practice, of our teaching, and of the prevention of errors, 
or inadequacies in our performance" (Duthie, 1993). 1 

Variability is reported in the delivery of virtually 
every aspect of health care. Often the care delivered 
seems inconsistent with available guidelines or current 
evidence. 2-4 Performance improvement has an important 
positive impact on the patient’s outcome. 5 Closing the 
knowledge-performance gap is thus considered one of the 
most important challenges for the medical profession in the 
forthcoming years. 6-8 Awareness is the first and crucial step 
in achieving this goal. In this respect, medical audit is of the 
utmost importance. 9 The aim of this study was to audit cur¬ 
rent medical care delivered to paediatric trauma patients 
in Flanders (Dutch-speaking northern part of Belgium) and 
identify possible areas susceptible for improvement. 

Methods 

The PENTA registry prospectively collected data on paedi¬ 
atric trauma patients in a representative sample (n = 18) of 
Flemish hospitals during 2005. 10 Ethical committee approval 
was obtained in each. Data were rendered anonymous 
before analysis. The data collection for children with severe 
trauma (length of hospitalisation >48 h; n = 244) included 
291 variables providing detailed information about trauma 
circumstances, injury severity, patient status, and medical 
care delivered within the first hours after trauma (includ¬ 
ing pre-hospital care). All cases with an injury severity 
score [ISS] of >13 (n = 96) were withheld for further eval¬ 
uation by panel review. 11 Two trained experts (PVDV, MD 
and MS, MD-Ph.D.) independently reviewed cases and then 
discussed each case in turn during several consensus meet¬ 
ings. Each reviewer received a short case description about 
the patient’s care trajectory, severity of injuries and out¬ 
come, as well as the complete original chronological data 
file. To facilitate the audit process, we also provided a (not- 
limitative) list of audit filters (i.e. potential problem areas) 
(see Table 1 ). Reviewers were instructed to base their eval¬ 
uation upon existing evidence and current guidelines, as far 
as these were uncontested in any recent literature. 12-14 For 
instance, 'steroids in spinal injury’ or 'crystalloids versus 
colloids for resuscitation’ would not be considered, while 
'high-dose steroids in neurotrauma’ or 'a dextrose solu¬ 
tion IV for resuscitation’ would. An audit filter was then 
withheld as "default" only if it was considered avoidable 
and not justified by circumstances; if both reviewers abso¬ 
lutely agreed, and if the registered data were considered 
sufficient to evaluate the process at hand. Whenever there 
was any reasonable doubt, the audit filter was not with¬ 


held as default (in dubio, abstine). Initially, "defaults" 
were identified irrespective of their potential impact on 
the patient’s outcome. This impact on morbidity and mor¬ 
tality was afterwards scored separately as 'no impact’ or 
'some impact’ on the outcome of that particular patient. 
To optimise the validity of the results, interobserver vari¬ 
ability was assessed by reviewing 26 cases a second time 
with two other independent experts (PC, MD-Ph.D. and 
SI, MD-Ph.D.). Twenty-two cases were selected at ran¬ 
dom (Statsdirect©). 15 Two cases were added because one 
expert of the first panel was to some extent involved in the 
medical care provided; two others because of the case com¬ 
plexity. The modalities of the second review process were 
similar to the first. After their initial review, the second 
panel was informed about the defaults withheld by the first 
panel. Subsequently, both panels discussed the 26 cases to 
reach a final consensus. For agreement between reviewers 
and panels, crude agreement (in %) as well as change- 
corrected agreement (Cohen’s kappa k) was calculated using 
Statsdirect©. 15 

Results 

Of the 96 identified cases, 92 were withheld for evaluation 
by the first panel. Two were excluded because of insuffi¬ 
cient data; two others because they died before any medical 
intervention. The median ISS was 21 (interquartile range 
[IQR] 15), the median length of stay in hospital was 9 days 
(IQR16). Fifty cases were vehicle-related accidents, 20 were 
falls. Approximately 70% received pre-hospital care and 78% 
were admitted to intensive care. The majority (75%) of cases 
had a good outcome, 11 children showed moderate to severe 
disability at discharge and 12 died. 16 

The first panel identified 264 defaults in the 92 stud¬ 
ied cases. Only in four patients were no defaults withheld. 
In 15 cases there was only one default, in 28 cases two, 
in 21 cases three, and in as much as 24 cases there were 
four or more defaults identified. Defaults were recognised 
in the pre-hospital phase (30%), as well as in the emergency 
department (43%), or during the first hours after hospitali¬ 
sation including during early surgery (27%). 

The interobserver variability in the first panel was low 
with an overall crude agreement before discussion of 75.2% 
(Cohen’s k 0.50). Of the 264 defaults withheld by the first 
panel after consensus, 88% had been recognised by both 
reviewers independently beforehand. 

One in four defaults (25.4%) was thought to have had 
a certain impact on the patient’s outcome. Of all these 
latter defaults, about half were related to "ABC" manage¬ 
ment (airway, breathing, circulation). Figure 1 specifies the 
number of defaults recognised in different areas of medi¬ 
cal care. Overall, we observed problems with analgesia in 
as much as 27 of 89 relevant cases (30%), problems with 
cervical spine management in 18 of 82 (22%), suboptimal 
or inadequate oxygenation and ventilation in 38 of 92 (41%) 
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Table 1 A not-limitative list of audit filters (i.e. potential problem areas) 

• On-scene time and transport times 

• Triage; appropriateness of referral 

• Time spent in the emergency department 

• Timing of and indications for radiology 

• Time to abdominal surgery; more specifically in case of hypotension because of intraabdominal pathology 

• Time to f 'urgent” vascular surgery; trauma surgery and neurosurgery (specifically for epidural haematoma) 

• Appropriateness of surgery indication (including damage control surgery) 

• Fracture immobilisation/fixation 

• Fluid management (type, amount)/IV access - perfusion pressure 

• Oxygen therapy - oxygenation 

• pC0 2 management - ventilation, specifically in case of traumatic brain injury 

• Airway management/intubation, specifically in patients with coma (GCS < 8) 

• Temperature management, specifically in case of traumatic brain injury 

• In patients with GCS< 13: timing of CT scan, cervical spine immobilisation, monitoring 

• Metabolic derangements (glucose, sodium...), specifically in case of traumatic brain injury 

• Burn care 

• Missed, incorrect or delayed diagnosis; procedures; including CPR 

• Adequacy of materials, procedures, medication doses in view of the age of the patient 

• Analgesia and sedation 

• Donor selection/management 

• Resource use; length of stay 


and defaults in circulatory management in 29 of 92 patients 
(31%) (see also Figure 2). Unacceptable hypo- or hypercapnia 
was encountered in 14 of 66 cases (21%) at risk for intracra¬ 
nial hypertension. 

In the 26 cases selected for second review, the agree¬ 
ment between both panels before discussion was found to be 
'good’ (crude agreement 74.8%; Cohen’s k 0.63). The agree¬ 
ment was, on the other hand, only 'fair’ when considering 
the presumed impact on the individual patient’s outcome 
(crude agreement 58.3%, Cohen’s k 0.20). After the final 
consensus meeting between both panels, 103 defaults were 
withheld. Of the 98 defaults recognised by the first panel, 12 
were no longer withheld after final consensus, yet 17 others 
had been added. 


Discussion 

A need for audit? 

It is the responsibility of the medical profession to evaluate 
the quality of care and to strive for continuous performance 
improvement. This has little to do with sorting out ''bad 
apples”, but everything to do with systematic and con¬ 
tinual analysis of input and process of care, as well as 
outcome; with the intent to reduce variation in care and 
eventually improve patient satisfaction and outcome. 17,18 In 
this respect, outcome evaluation has become an inherent 
part of most research initiatives and to a lesser extent of 
routine clinical care. Assigning a specific outcome to a par- 
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Figure 1 The absolute number of defaults (n = 264) recognised in different areas of medical care. 
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Figure 2 Numbers of patients with (light) and without (dark) a default for particular areas of medical care. For the evaluation of 
each area of care only relevant cases were included (light + dark, together 100%). 


ticular intervention is, however, often difficult because of 
time effects and intervening events. Further, outcome mea¬ 
sures are often prone to bias and may lack responsiveness 
with regard to system improvements. 1 Finally, bad outcome 
is relatively rare, so that studies trying to show important 
correlations are most often underpowered. 

Process measures are, although not always strong pre¬ 
dictors of outcome, clearly more controllable and more 
responsive to change. 19-21 However, the reliability and valid¬ 
ity of the methodology has been questioned. Often, it is 
based upon implicit judgments by an expert panel which are 
inherently subjective with low reproducibility and not very 
well defined end-points. 7,22,23 Still, while lacking in quanti¬ 
tative precision, process evaluation has the ability to detect 
opportunities for improvement and to serve as a major impe¬ 
tus for necessary changes. 17,24,25 Historically, most of these 
studies evaluated preventability of mortality or morbidity. In 
our opinion, whether or not suboptimal care leads to certain 
morbidity or mortality is of secondary importance. Each of 
these 'defaults in care’ has the potential of generating addi¬ 
tional morbidity or mortality and should be acknowledged as 
such ('near-miss principle’). Although guidelines are often 
not based on much evidence and too strict adherence to 
existing guidelines may even obstruct evolution in medicine, 
it is without discussion that certain deviations from 'stan¬ 
dard care’ are indeed 'inappropriate’ care. Identifying these 
clear deviations and the reasons for them is one of the main 
goals of a panel evaluation. In our audit, evaluation was 
based on widely accepted guidelines, as far as they were 
uncontested in recent literature. 12-14 


The validity of the methodology 

The value of any audit is directly related to the availability 
of accurate and complete data. We collected these data as 
part of the PENTA registry in Flanders, a population-based 
paediatric trauma registry. 16 Data collection was exhaus¬ 


tive and we invested a lot of energy into data accuracy and 
completeness. Our study had, as has every audit, a certain 
resolution, related to the method of collection and the vari¬ 
ables collected. We did not have sufficient information on 
(and could therefore not evaluate), e.g. treatment effects 
(pain relief after analgesia...), adequacy of technical pro¬ 
cedures or communication.... 

Importantly, reviewers focussed only on clear 
"defaults”; on significant violations of basic care princi¬ 
ples, not on minor infractions. If there was any reasonable 
doubt, that default was not withheld as such (in dubio, 
abstine). Panel members all had ample expertise and 
received specific training regarding the review process. 
Final decisions were taken in full consensus. Interrater 
variability was assessed by presenting 26 cases to a second 
independent panel. The two cases where one member 
of the first panel was involved were eventually withheld 
because a lot of the care provided was not related to 
the reviewer. As only 22 of the 26 cases were selected at 
random, a bias in favour of disagreement was introduced. 
Still, the agreement between both panels (also in the 
not-random cases) for defaults recognised, even before 
discussion, was clearly 'good’. The agreement before 
discussion regarding the impact on outcome was, on the 
other hand, far less. Linking a recognised default with the 
outcome of an individual patient often turned out to be 
difficult (as we have already pointed out in the beginning 
of the discussion section). 

Although the above data confirm the inherent degree 
of interrater variability associated with panel review, 
we feel it does not diminish the importance of its 
conclusions. 


The importance of our findings 

It is important to place our findings in the right perspective. 
For every default, several other actions were timely and 
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appropriate. On the other hand, a bias might also have been 
introduced because hospitals participating in our registry (18 
of 71 Flemish acute care hospitals) tended to be larger and 
had an active interest in paediatric trauma. 

The majority of the detected defaults were observed 
in basic areas of advanced paediatric life support (e.g. 
airway and cervical spine management, oxygenation, ven¬ 
tilation, fluid resuscitation, pain relief), in part because of 
the focus of this audit. These findings are not unique to 
trauma nor are they limited to Flanders. Several recent stud¬ 
ies similarly show an alarmingly high degree of inappropriate 
care, emphasising the need for new ways of educa¬ 
tion, process evaluation and performance improvement 
initiatives. 25-29 Although widely used, passive dissemination 
of knowledge seems ineffective in improving actual medi¬ 
cal performance. 8 More performance-based teaching has a 
positive effect on perceived self-efficacy and overall skills, 
although the evidence in favour is still limited. 30-32 Given 
the fact that paediatric emergencies are rare, it is likely 
that this positive effect diminishes over time if training is 
not repeated regularly. 33 ’ 34 Broster et al. showed that for a 
large region in the UK the majority of paediatric consultants 
and registrars, although at one point having attended an ALS 
course, did not keep up with maintaining/recertifying these 
skills. 35 This situation is very similar for Flanders, where 
there is actually no incentive whatsoever for following, e.g. 
an APLS/EPLS course. 

It is clear that there are no easy solutions to the 
existing knowledge-performance gap. However, as several 
authors agree, on-going assessment of actual medical prac¬ 
tice is undoubtedly the first step. 6 ’ 7 ’ 20 ’ 36 A further step could 
then be not only to implement a more performance-based 
approach to teaching but also to ensure regular recertifi¬ 
cation in view of the low rate of exposure to paediatric 
emergencies for most physicians. 


Conclusions 

We audited medical management in severe paediatric 
trauma victims in Flanders and identified several problem 
areas. Defaults were often recognised in basic areas of 
advanced paediatric life support (ABCD management). The 
inherent degree of interrater variability associated with 
panel review does not, in our opinion, diminish the impor¬ 
tance of these findings. More performance-based teaching 
and timely recertification could be a more efficient way 
of providing physicians with the necessary skills to manage 
paediatric emergencies. 
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Appendix A 

The following Flemish hospitals participated in the registry: 
ASZ Aalst, St. Blasius Dendermonde, UZ Gent, AZ MMSJ Gent, 
Jan Palfijn ZH Gent, AZ St. Lucas Gent, AZ Oudenaarde, AZ 
MM St. Niklaas, AZ Groeninge Kortrijk, HH ZH Menen, HH ZH 
Roeselare, AZ St. Maarten Mechelen, AZ St. Maarten Duffel, 
St. Vincentius ZH Antwerpen, Klina Brasschaat, AZ Stuiven- 
berg Antwerpen, ZH Noord-Limburg Lommel, ZOL Genk, UZ 
Gasthuisberg Leuven. 

PENTA was the result of the combined effort of a lot 
of people in each hospital. We would like to thank: Mevr. 
Elena Kourdiokova, Dr. Vergote L, Dr. de Leeuw, Mevr. De 
Boeck I, Dr. Vandenbroucke P, Dr. Van Laeken K, Dr. Van 
Bosstraeten, Dhr. Van Den Steen A, Dr. Verrijckt A, Dhr. Fort- 
ain P, Dhr. JonckersM, Mevr. Van derGucht A, Dr. Plasschaert 

F, Prof. Buylaert W, Mevr. Goedertier H, Dr. Kamoen K, Dr. 
De Somer A, Dhr. Roberti A, Dr. Vercruysse T, Dr. De Graeve, 
Dhr. Goossens J, Dhr. Van Quaethem, Dr. Dewilde B, Dr. Van 
Der Plaetsen M, Dr. Snoeck, Dhr. De Jonghe E, Dr. Damen 
J, Dr. Van Tichelen J, Dr. Putzeys G, Dr. Struyve, Dhr. Gunst 

G, Mevr. Herpoel R, Dr. Demey R, Dr. Coppe E, Dhr. Delie 
C, Dhr. Seynaeve W, Dr. Spoelders K, Dr.Gijsenbergh F, Dhr. 
Gijbels L, Dr. Vertruyen A, Dr. Druwe, Dhr. Van De Heijning 
L, Dr. Mortelmans L, Dr. Smets G, Dr. Vundelinckx, Dhr. De 
Houer J, Mevr. Smeets K, Dr. Claessens Y, Dhr. HendrickxM.l, 
Mevr. Vereecken H and all others who helped with collecting, 
retrieving etc. 
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